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SUMMARY 

In this paper, we study state-feedback control of Markov jump linear systems with partial information. In 
particular, we assume that the controller can only access the mode signals according to a hidden-Markov 
observation process. Our formulation generalizes various relevant cases previously studied in the literature 
on Markov jump linear systems, such as the cases with perfect information, no information, and cluster 
observations of the mode signals. In this context, we propose a Linear Matrix Inequalities (LMI) formulation 
to design feedback control laws for (stochastic) stabilization, H 2 , and Hca control of discrete-time Markov 
jump linear systems under hidden-Markovian observations of the mode signals. We conclude by illustrating 
our results with some numerical examples. Copyright © 2010 John Wiley & Sons, Ltd. 
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1. INTRODUCTION 

Markov jump linear systems [1] are an important class of switched systems in which the mode 
signal, responsible for controlling the switch among dynamic modes, is modeled by a time- 
homogeneous Markov process. This type of systems has been widely used in multiple applications, 
such as robotics [2, 3], economics [4], networked control [5], and epidemiology [6]. Solutions to 
standard controller synthesis problems for Markov jump linear systems, such as state-feedback 
stabilization, quadratic optimal control, H 2 optimal control, and iJoo optimal control (see, e.g., the 
monograph [1]), can be found in the literature. These works, however, are based on the unrealistic 
assumption that the controller has full knowledge about the mode signal at any time instant. 

To overcome this limitation, several papers investigate the effect of limited and/or uncertain 
knowledge about the mode signal. For example, the authors in [7] studied H 2 control of discrete¬ 
time Markov jump linear systems when the state space of the mode signal is partitioned into subsets, 
called clusters, and the controller only knows in which cluster the mode signal is at a given time. 
Similar studies in the context of iToo control can be found in [8, 9]. In the extreme case of having a 
single mode cluster (in other words, when one cannot observe the mode), the authors in [10, 11] 
investigated quadratic optimal control problems. Most of the above works can be studied in a 
framework based on random and uncertain mode observations (see [12] for the description of this 
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framework in the context of H 2 control). In a complementary line of work, we find some papers 
assuming that the mode signal can only be observed at particular sampling times, instead of at any 
time instant. In this direction, we find in the literature a variety of random sampling strategies of 
the mode signal. The authors in [13] designed almost-surely stabilizing state-feedback gains when 
the sampling times follow a renewal process. Similarly, the authors in [14, 15] derived stabilizing 
state-feedback gains using Lyapunov-like functions under periodic observations. 

In this paper, we propose a framework to design state-feedback controllers for discrete-time 
Markov jump linear systems assuming that the mode signal can only be observed when a Markov 
chain (different than the one describing the mode signal) visits a particular subset of its state 
space. We call this observation process hidden-Markov, due to its similimde with hidden-Markov 
processes [16]. We show how hidden-Markov observation processes generalize many relevant cases 
previously studied in the literature, such as those in [7, 8, 12-15]. In this context, we propose 
a Linear Matrix Inequalities (LMI) formulation to design feedback control laws for (stochastic) 
stabilization, H 2 , and Hoc control of discrete-time Markov jump linear systems under hidden- 
Markov observations of the mode signal. It is important to remark that, since the observation process 
is hidden-Markovian, existing control synthesis methods for Markov jump linear systems, such as 
those in [1,7,8], do not apply to our case. 

The paper is organized as follows. In Section 2, we formulate the state-feedback control problem 
for Markov jump linear systems with hidden-Markovian observations of the mode signal. We show 
in Section 3 that the resulting closed-loop system can be reduced to a standard Markov jump linear 
system by embedding the (possibly non-Markovian) stochastic processes relevant to the controller 
into an extended Markov chain. In Section 4, we derive an LMI formulation to design state-feedback 
gains for stabilization, H 2 , and iJoo control problems. Finally, in Section 5, we illustrate our results 
with some numerical examples. 

Notation 

The notation used in this paper is standard. Let N denote the set of nonnegative integers. Let M” 
and denote the vector spaces of real n-vectors and nx m matrices, respectively. By Ij-H, 

we denote the Euclidean norm on M". Pr( ) will be used to denote the probability of an event. 
The probability of an event conditional on another event A is denoted by Pr(- | A). Expectations 
are denoted by E[-]. Eor a positive integer N, we define the set [N] = {!,..., N}. Eor a positive 
integer T and an integer k, define [fcj t as the unique integer in {0,...,T — 1} such that k— [fcj t 
is an integer multiple of T. When a real symmetric matrix A is positive (resp., negative) definite, we 
write A > 0 (resp., A < 0). The notations A > 0 and A < 0 are then understood in the obvious way. 
Eor sets of matrices A = {Aa}aga C and B = {BaIaga C sharing the same index 

set A, we define another set of matrices AB = {AaBa}aga C The symbol * will be used 

to denote the symmetric blocks of partitioned symmetric matrices. Einally, indicator functions are 
denoted by l(-). 


2. PROBEEM EORMUEATION 

In this section, we formulate the problems under study. Eet n, m, q, £, and N be positive integers. 
Eor each i e [N], let G B, G C, G A G and E, G Also, 

let r = {r(fc)}^g be the time-homogeneous Markov chain taking values in [TV] and having the 
transition probability matrix P G Consider the Markov jump linear system [1]: 

^ r a;(/c -f 1) = A,.(fc)a:(fc) -f Br(k)u{k) + E^(k)w{k), 

\ z{k) = Cr(k)X{k) + A(fe)W(^)- 

We call X and r the state and the mode of E, respectively. The signal w represents an exogenous 
disturbance, u is the control input, and z is the measured signal. The initial conditions are denoted 
by a;(0) = xq and r(0) = tq. We will assume that xq and tq are either deterministic constants or 
random variables, depending on the particular control problems considered. 
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2.1. State-feedback control with hidden-mode observation 

In this paper, we consider the situation where the controller cannot measure the mode signal at 
every time instant. To study this case, we model the times at which the controller can observe the 
mode by the stochastic process t = taking values in N U {oo}. We call t the observation 

process and each ti an observation time. For each i, we assume either ti < ti+i or U = = oo. 

It is understood that, if U < = oo, then no observation will be performed after time U. 

In this paper, we focus on the following class of observation processes: 

Definition 2.1 

We say that an observation process t is hidden-Markov^ if there exist an M e N, a Markov chain 
s = {s(fc)}fc>o taking values in [M] (independent of the mode r), and a function /: [M] —> {0,1} 
such that 


fo = min{fc > 0 : f{s{k)) = 1} 


and, for every i > 0, 


ti+i = min{fc > U : f{s{k)) = 1}, 


where the minimum of the empty set is understood to be oo. 

For example, if the image of / equals the set {1}, then the controller observes the mode at all 
time instants. On the other hand, if / maps into {0}, then the controller never observes the mode 
signal. In fact, the class of hidden-Markov observation processes contains many other interesting 
examples as will be seen below. Throughout the paper, we denote the transition probability matrix 
of s by Q G In what follows, we provide three particular examples that can be formulated 

as hidden-Markovian observation processes: 

Example 2.2 (Gilbert-Elliot channel) 

Consider the case where the controller observes the mode through a Gilbert-Elliot channel [17]. This 
channel has two possible states: the good (G) and bad (B) states. When the channel is at the G state, 
it transmits the mode signal to the controller; in contrast, when it is at state B, it does not transmit. 
This channel switches its state according to a Markov chain, defined as follows. Eet p, g S [0,1] be 
the transition probabilities from G to B and B to G, respectively. We can formulate this channel as a 
hidden-Markovian observation process (Definition 2.1) using the following parameters: 



M = 2 


Our second example is closely related to the observation processes investigated in [12]: 

Example 2.3 (Observations with independent and identically distributed failures) 

Assume that, at each time instant, the controller attempts to observe the mode signal but it fails with 
probability pj G [0,1], independently from the observations at other time instants. This observation 
process can be implemented as a hidden-Markovian observation process using the following 
parameters: 



We remark that, under a similar problem setting, the authors in [12] propose a framework for 
stochastic stabilization and H 2 control of Markov jump linear systems. 

Our last example is concerned with periodic observation with failures: 


tWe adopt the terminology “hidden-Markov” because the process {/(s(A:))}fe>o characterizing the observation process t 
is a hidden-Markov process [16]. 
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Example 2.4 (Periodic observation with failures) 
Let £ be a positive integer and p G [0,1]. Dehne 



; 1 



1 1 


M = £ + l, Q = 

1 

1 

1 


p 1 — p ' 



, /(A) 


1, ifA = l, 
0 , otherwise. 


Then, we can see that ti+i —ti is a. positive integer multiple of £ with probability one and, also, 
Pr(fi_|_i — ti = k£) = (1 — p)^~^p for alH > 0 and fc > 1. The corresponding observation process 
describes the situation where the controller tries to observe the mode signal every i time units with a 
probability of success p for each observation. In particular, for p = 1, this observation process gives 
the periodic case considered in [14,15]. 


In order to specify the behavior of the controller between two consecutive observation times, 
we introduce the following processes. Given an observation process t, we define the stochastic 
process r = {r(fc)}^p by 


T{k) 


max{<i '■ ti < k, i > 0}, 

To, 


if k > to, 
otherwise. 


where tq is an integer satisfying 

I To = 0 , if to = 0 , 

1 To < 0 , otherwise. 

For each time k, the above dehned T{k) represents the most recent time the controller observed 
the mode. We, in particular, have T{ti) = ti for every z > 0. Notice that, for k < t^, we augment 
the process r with a negative integer tq. This is because, before time k = to, no observation is 
performed by the controller yet. This augmentation is not needed if to = 0, in which case we set 
To = 0 as in ( 1 ). 

We also define the stochastic process a = {cr{k)}^^ taking values in [N] by 


cr(fc) 


r{T{k)), if fc > to, 

uo, otherwise. 


where cjo is an element in [TV] satisfying 


[fo = 0] ^ [cto = ro]. (2) 

For each k, the random variable a{k) represents the most-updated information about the mode 
signal kept by the controller at time k. We again notice that, by the same reason indicated above, cr 
is augmented by an arbitrary cto before the time instant k = to (i.e., before the first observation is 
performed). As is the case for tq, if to = 0, then this augmentation is not needed and thus we set 
CTo = To as in (2). See Figure 1 for an illustration of the stochastic processes described so far. 

In what follows, we present the state-feedback control scheme studied in this paper. We assume 
that the controller has an access to the following pieces of information at each time k> 0: 
(i) the state variable x{k), (ii) the most recent observation a{k) of the mode r, and (Hi) the 
quantity k — T{k), which is the time elapsed since the last observation. Specihcally, the state- 
feedback controller under consideration takes the form 


llik) [fc —r(/c)J T + 

where S for each 7 G [N] and 5 G [T]. The first subindex of K in (3), i.e., a{k), allows 

the gain to be reset whenever the controller observes the mode. The second subindex^ of K in (3) 


tNotice that we add 1 to [fc — r(A:)Jy in the second subindex of K in (3) to make the index 5 of start from 1, 
instead of 0. 
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Figure 1. An observation of the mode r. The observation times to, ti, t 2 , ■ ■ ■ are determined by the Markov 
chain s and the function /: [3] —> {0,1} given by f{s) = 1 if s G A = {2} and /(s) = 0 otherwise. Until 
the first observation time to = 2, the most recent observation cr is temporarily set to ao = 2. 


cr(fc)“ 

20 

1 


allows the controller to change its feedback gain between two consecutive observation times with 
period T as in [15], rather than keeping them to be constant. We will later see in Section 5 that, as 
the period T increases, the performance of the controller can in fact improve. Throughout the paper, 
we will use the notation 

p{k) = \k- T[k)\T + 1 - 

Notice that the initial condition of p is given by / 9 ( 0 ) = po= [—to\t + 1- 


2.2. Performance measures 

We now introduce several performance measures used to evaluate the state-feedback control law (3). 
The feedback control law (3) applied to S yields the following closed-loop system 


Sif : 


x[k “El) x{Jf) -\- 

z{JP) = (Cr(k) “E ^r{k)^a{k),p{k)) x{kf 


Let us introduce the compact notation 


r{k) = [r(k),s{k),a{k),p{k)). (4) 

Also, define X as the set of quadruples (a, /3, 7 , 6) G [N] x [M] x [N] x [T] such that, if /(/3) = 1, 
then a = j and 5=1. The set X contains all possible values that can be taken by the stochastic 
process f. We denote the initial condition for f as r (0) = Tq. We sometimes denote the trajectories x 
and z ofEx by x(-; xq, rg, w) and z(-; xg, Tg, tn), respectively, whenever we need to clarify the initial 
conditions as well as the disturbance w. We finally remark that, by the conditions in (1) and (2), fg 
is determined by rg, sg, erg, and po as 


ro = 


(5) 
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The first performance measure under consideration is mean square stability: 

Definition 2.5 (Mean square stability) 

We say that Ek is mean square stable if there exist C > 0 and A S [0,1) such that £'[||a;(fc)|p] < 
CA'^IIxoII^ for all xq, ro, and A: > 0, provided w = 0. 

In order to define the second performance under consideration, we first need to introduce the 
space of square summable stochastic processes, as follows. Let 0^ be the cr-algebra generated by the 
random variables {r(fc), s(A:),... , r(0), s(0)}. Define (£^ for short) as the space of M”-valued 

stochastic processes / = {f(k)}k>o such that f(k) is an M”-valued and 0fc-measurable random 
variable each fc > 0 and, moreover, i?[||/(fc)|p] is finite. For / s £^, define its A^-norm ||/||2 

by ll/lll = i?[||/(A:)|p]. Then, we extend the definition of the JI 2 norm of a Markov jump 

linear system introduced in [7], as follows: 

Definition 2.6 (H 2 norm) 

Assume that fg follows the probability distribution fl. Define the H 2 norm of 52k by 



where d denotes the ith standard unit vector in and is the function defined on N by (^( 0 ) = 1 
and (j){k) = 0 for fc > 1 . 

Our third and last performance measure is the Hoc, norm. In our context, we use the following 
definition, which is an extension of the one for standard Markov jump linear systems [18]: 

Definition 2 .7 (i7oo norm) 

Assume that 52k is mean square stable and xg = 0. Define the Hoc, norm of 52k by 


||Sif||oo = sup 


sup sup T]—iT- 
foGXi«er2(R9)\{o} ll^lb 


Having introduced these three performance measures, we can now formulate the problems under 
consideration. The first problem is concerned with stochastic stabilization, which is stated as 
follows: 

Problem 2.8 (Stabilization) 

Find a set of matrices {iTg,_ 5 }g,g[ 7 v], 5 G[T] C such that 52k is mean square stable. 

The second problem is concerned with the stabilization of the closed-loop system with an upper- 
bound on the H 2 norm. In this problem, we assume that the distributions of rg and sg are given. 
Thus, the parameters to be designed are feedback gains K and the distribution u of the pair (erg, pg): 

Problem 2.9 (H 2 control) 

Assume that the distributions of rg and sg are known. For a given 7 > 0, find a set of matrices 
{Arg,_ 5 }.yg[Af] C and a distribution v such that 52k is mean square stable and ||Sir ||2 < 7- 

Remark 2.10 

Given the distribution (resp., of rg (resp., sg), using (5) we can find p as 


Pr{a)ps{fi) if/(/3) = l, 

Pr{a)ps{fi)u{'y,S) otherwise, 


m(x) 


( 6 ) 


for every x = (a, (3, 7 ,5) € X. 

The last problem is the stabilization of the closed-loop system with an upper-bound on the TToo 
norm: 


Copyright © 2010 John Wiley & Sons, Ltd. 
Prepared using rncauth.cis 


Ini. J. Robust. Nonlinear Control (2010) 
DOI: 10.1002/rnc 





STATE FEEDBACK CONTROL WITH HIDDEN-MARKOV MODE OBSERVATION 


7 


Problem 2.11 {Hoo control) 

For a given 7 > 0, find a set of matrices C such that Hk is mean square 

stable and ||oo < 7- 

We remark that T,k is no longer a standard Markov jump linear system, due to the nature of 
the processes a and p. Therefore, we cannot use any of the techniques in the literature [1,7, 8] to 
synthesize a control law. In fact, we cannot use existing techniques even to analyze the performance 
of Yik- Moreover, due to the generality of hidden-Markov observation processes, we cannot use any 
of the results in [12, 13, 15], recently proposed to design state-feedback control laws for Markov 
jump linear systems with partial mode observation. 


3. ANALYSIS OF CLOSED-LOOP SYSTEM 


In this section, we show how to analyze the closed-loop system In this direction, we reduce 
Yik to a standard Markov jump linear system by embedding stochastic processes appearing in the 
closed-loop system (which are not necessarily Markovian) into an extended Markov chain with a 
larger state space. Eet us begin with the following observation: 

Lemma 3.1 

The stochastic process r defined by (4) is a time-homogeneous Markov chain. Moreover, its 
transition probabilities are given by 


Pr(r(fc + 1) =x' \ r{k) = x) 


l(a' =x',S' = l)Paa'<?/3/3', if f(/3') = 1, 

1 ( 7 ' = 5 - IJt = 0)Paa'qi3p', otherwise. 


for all X = and x' = (a',/?', 7 ',5') in X. 

Proof 

Eet fco G N, /c > fco, and Xi = (cti, /3i, 7 i, 6i) G X (i = ko, ■■■, k + 1) he arbitrary. Eor each i, define 
the events Ai and Bi as Ai = {f(i) = xa ■ • • ? ''"(^o) = Xfeol ^tid B^ = {f(i) = Xi}- LFnder the 
assumption that Ak is not the null set, we need to evaluate the conditional probability 


Pr(r(fc + 1) = Xfc-ri I A) = Pr(Afc+i)/Pr(Afc). 
Remark that, since Ai f 0, we have 


( 8 ) 


(T{k) = 7fe, p{k) = 6k. (9) 

Pirst, assume that f{f3k+i) = 1- Then, by Definition 2.1, an observation occurs at time fc -I-1, i.e., 
we have T{k -F 1) = /c -F 1 and a{k + 1) = r{k + 1). This implies that 

Bk+i = {r{k + 1) = a/c+i, s(fc -F 1) = l3k+i,ak+i = Jk+i, 1 = 4-i-i}- 

Therefore, since Ak+i = AkL Bk+i, 

Ak+i = {ofc+i = 7fc+i,4-ri = 1} n {r{k + 1) = ak+i,s{k -F 1) = /3fc+i} C Ak (10) 


and hence 


Pr(7lfc+i) = l(afe+i = 7fe+i, 4-1-1 = l)Pr({r(fc -F 1) = ak+i,s{k + 1) = (3k+i) C Ak). (11) 
The probability appearing in the last term of this equation can be computed as 

Pr({r(fc -F 1) = ak+i,s{k + 1) = 4-i-i} C Ak) 

= Pi{Ak)Pr{r{k + 1) = ak+i,s{k + 1) = 4+1 I A) 

= Pr{Ak)Pr{r{k + 1) = ak+i,s{k + 1) = 4+i I r{k) = ak,s{k) = /3k) 

= Pr{Ak)Pakak + iq/3kl3k + l ! 
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where we have used the fact that both r and s are time-homogeneous Markov chains. Thus, from 
equations (8), (11), and (12), we conclude that for the case of f{(3k+i) = 1, 

Pr(f(fc + 1) = Xk+l I Ak) = l(afe+l = Jk+l,Sk+l = l)Pafcafc+i<?/3fc/3fc+i- (13) 

Second, consider the case where f{f3k+i) = 0. In this case, the Markov mode r is not observed at 
time fc + 1, hence, we have T{k + 1) = T{k) and a{k + 1) = a{k). Therefore, using equations (9), 
in the same way as we derived (10), we can show that 

Ak+i = {7fc+i = 7fe, [h+i -5k- IJt = 0} n {r{k + 1) = au+i, s{k + 1) = /3fe+i} n Ak (14) 

and hence 

Pr(A+i) = l(7fc+i = 7fcj L4+1-4 - IJr = 0)Pr({r(fc + 1) = ak+i,s{k + 1) = 4+i} n Ak). 

Therefore, from equations (8), (12), and (14), we show that, if f{j3k+i) = then 

Pr(f(/c + 1) = Xk+i I Ak) = l(7fc = 7fc+i, L4+1 - 4 - IJt = ^)Pa^,ak+lmkPk+l■ (15) 

Since the probabilities (13) and (15) do not depend on fco, letting ko = k and feg = 0 in (13) and 
(15), we obtain 

Pr(f(/c + 1) = Xfc+i I r{k) = Xfc, ■ ■ •, r{ko) = Xko) = Pr(f(fc + 1) = Xk+i \ r{k) = Xk) 

for every A: > 0. This shows that f is a Markov chain since Xfco) ■ ■ ■ )Xfc+i G are arbitrary. 
Moreover, since the probabilities (13) and (15) do not depend on k, we conclude that the Markov 
chain f is time-homogeneous and its transition probabilities are given by (7). □ 

Lemma 3.1 states that the closed-loop can be represented as a Markov jump linear system 
with its mode being the extended Markov chain f. This observation leads us to the following 
definitions. For x, x' € X, we denote the transition probabilities of the Markov chain f (eq. (7) in 
Lemma 3.1) by = Pi{f{k -I-1) = x4 ^i^) = x)- Then, we introduce the Markov jump linear 
system 

- {x{k + l)= AKfi(k)x{k) + EK,e(k)w{k), 

^ \ z(k) = CK.g(k)xik), 

where 6 is the time-homogeneous Markov chain taking values in X whose transition probabilities 
are Pr(0(A: -}-1) = x' | 0{k) = x) = Pxx'^ ^he matrices Ak,x’ Ck,x^ and Ek,x are defined by 

Ak^x — Aa 4“ ,5, Ckc^x — 4a 4“ Ekc^^ — 7?^^, (16) 

for each x = (a, ,8,7, G X. We sometimes denote x and z by x{--,xq,9q,'w) and z{-\xo,9q,w) 
whenever we need to clarify initial conditions and disturbances w. 

The next corollary of Lemma 3.1 plays the key role in this paper. 

Corollary 3.2 

Assume that xq = xq, w and w have the same probability distribution, and fo and 9o have 
the same probability distribution. Then, the stochastic processes a:(-;a;o,ro,R') and x{--,xo,9o,'w) 
have the same probability distribution. Also, under the same assumption, the stochastic 
processes z(-; xq, tq, w) and z(-; xq, 9o, w) have the same probability distribution. 

Proof 

By the assumption, the Markov chains r and 9 have the same initial distribution. The chains also 
have the same transition probabilities from Lemma 3.1 and the definition of 9. Therefore, f and 9 
have the same probability distribution. Also, we notice that, by the definition of the matrices in (16), 
the system admits the following representation: 

x{k 4- 1) = AK^f{k)x{k) 4- Ef(k)wik), 
z{k) = CK,f(k)x{k). 

Therefore, Yk has the same dynamics as Yk. In conclusion, the claim holds true under the 
assumptions stated in the corollary. □ 
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Using Corollary 3.2, we can characterize the performance measures of the closed-loop 
system The next proposition provides a characterization for mean square stability: 

Proposition 3.3 

For i? = {i?x}xGa: C and x G define = Ex'gxPx'x-^x'G Then, the 

following statements are equivalent: 

1. T,k is mean square stable; 

2. T,k is mean square stable; 

3. There exist positive-definite matrices G for every y G X, such that — 

V^{AkQAI) > 0 . 


Proof 

The equivalence [2 3] immediately follows from the standard theory of Markov jump 

linear systems (see, e.g., [1]). Let us prove [2 1]. Assume that is mean square 

stable. In order to show that Y,k is mean square stable, let us take arbitrary xq G M" 
and To G X. Then, by Corollary 3.2 and the mean square stability of S^, we can show 
E[\\x(k; xq, fo, 0)|p] = E[\\x{k; xq, tq, 0)||^] < CA^UxolP for some C > 0 and A G [0,1), which 
implies mean square stability of T,k. We can prove [1 ^ 2] in the same way. □ 

The following proposition characterizes the H 2 norm of T,k' 

Proposition 3.4 

Let 7 > 0 be arbitrary. Assume that fg and 9^ follow the same distribution fl. Then, the following 
statements are equivalent: 

1. T,k is mean square stable and l|Srr||i<7; 

2. T,k is mean square stable and l|Srr||i<7; 

3. There exist a family of positive-definite matrices Qx G for every x G X, such that 

< T and V^{AkQAJ. + plEkE]^) < Q^, where the set pExE]. c 
I^nxn indexed by X is defined as {pEkE]^)^ = p{x)Ek,xE]^.x' 


Proof 

The proof of the equivalence [1 2] immediately follows from Corollary 3.2. Also, the 

equivalence [2 3] is a direct consequence of a standard result [7, Proposition 4] in the theory 

of Markov jump linear systems. The details are omitted. □ 


Finally, the following proposition characterizes the iFoo norm: 

Proposition 3.5 

For matrices Z = {^x,x'}x,x'ex C define Tx{Z) = XIx'gxPxx'-^x.x' G Let 7 > 0 

be arbitrary. Consider the following statements: 


1. Yjk is mean square stable and l|Srr||L<7; 

2. Eif is mean square stable and l|Srr||L<7; 

3. There exist matrices G M"^”, 77^ G M”^”, G M”^”, and ^^.x' G G X) 

such that 


^X + ^X ~ ^X 

'k 

k 

k 

0 

7 / 

k 

k 

^K,x^X 

Gk,x 

Hx + Hj-^xiZ) 

k 

Ck.xGx 

0 

0 

I 




> 0 , 


for all X, x' G X. 


Then, 1 and 2 are equivalent. Moreover, 3 implies 1 and 2. 
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Proof 

The implication [3 ^ 2] is a direct consequence of [8, Theorem 1]. Let us prove [1 2]. It 

is sufficient to show IlSifIloo = Let 0fc denote the cr-algebra generated by the random 

variables {0{k ),..., 0(0)}. Define P as the space of stochastic processes / = {/(fc)}^o such that 
-^[I1/(^)IP] is finite and, for each A: > 0, f{k) is an M”-valued and 0fe-measurable random 
variable. Define fhe norm of f G P by H/IH = Then, fhe Tfoo norm of Sk is 

given by 


llTliflloo 


sup sup 
eo6XiDGr2\{o} 


||z(-;O,0o,w)||2 

ll^lb 


To show that this norm equals ||Sj^||oo, it is sufficient to show that, if rg = 0o, then 


sup 

i«Gr2\{o} 


|z(-; 0 ,ro,w )||2 

Iklb 


= sup 


|z(-;O,0o,w)||2 

Iklb 


(17) 


By Corollary 3.2, the only difference between both sides of the equality (17) is the spaces and P. 
In other words, to complete the proof, it is sufficient to show that 0^, the cr-algebra generated by the 
random variables {r{k),s{k ),..., r( 0), s(0)}, coincides with the one generated by {r(fc),..., f(0)}. 
This is obvious because a{k) and p{k) are images of {r{k),s{k),r{k — 1), s(fc — 1),..., r(0), s(0)) 
under measurable functions. □ 


4. DESIGN OF FEEDBACK GAINS VIA EINEAR MATRIX INEQUAEITIES 


Based on the performance characterizations presented in the previous section, we now propose 
a formulation based on Einear Matrix Inequalities (EMI) to design feedback control laws for 
stabilization, H 2 , and Hao control of discrete-time Markov jump linear systems under hidden- 
Markovian observations of the mode signals. 

The next theorem provides an EMI formulation to solve the stabilization problem stated in 
Problem 2.8: 

Theorem 4.1 

Assume that the matrices G M”^”, G M”^”, and (j^ = (a,/ 3,7 ,5) g X) 

satisfy the linear matrix inequality 


G.,,s + Gls-T^xm 

for every x = (a, /3,7, 6) G X. For each 7 g [N] and S g [T], define 

K^,s = F^,sGf^,. 

Then, the resulting closed-loop system is mean square stable. 


(18) 


(19) 


Proof 

Assume that g G^^s G and F.y,s G satisfy (18), and define K by (19). Our 

proof is based on an argument proposed in [7]. Since X is a finite set, there exists an e > 0 such that 
= T>^{R) + el satisfies 


R^ 

^ * Gj^s + G^^g — 


( 20 ) 


where we have used AaG.y^s + BaF.y^s = Ak.xG-^, 5 - Since this inequality implies R^ > 0, we 
have Q^> el > 0. We here recall that, for a positive definite matrix A g and another 

matrix B g it holds that (see, e.g., [7]) BA~^B^ > B + B^ — A. Using this fact in (20), 

we obtain 


F-x 

'k 


GlsQf^G 


'X 


7,(5 


> 0 . 


( 21 ) 
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Also, from (20), we see that G '^^5 + G'^^>0 and therefore G-y^s is invertible. Hence, we 
can take the Schur complement of the positive-dehnite matrix in (21) with respect to 
to obtain — Ak,xQx^1< x ^ ^o this inequality the operator Vx, we obtain 

Qx — VxiAxQA]^) > el > 0. Therefore, by Proposition 3.3, S_r- is mean square stable. □ 

Secondly, the next theorem provides an LMl formulation to solve the H 2 control problem stated 
in Problem 2.9: 

Theorem 4.2 

Let 7 > 0 be arbitrary. Assume that Wx € Rx € M"^”, Fy^s G Gy^s S M"^”, and 

1 ^( 7 , > 0 (x = (a, /3, 7 , 6) e X) satisfy the following linear matrix inequalities 

Rx A^Gy^s F BaFy s 

Gy,s + Gls-^xiR)\ ’ 

Wx GaGy^S + D^Fy S „ 

Gy^s + Gls-'^xiR)] ^ 

E < 7, 

xex 
N T 

= 1 , 

7 — 15—1 

for every x = (a, /?, 7, (i) G X. Define the feedback matrix K by (19). Then, the closed-loop system 
Sif is mean square stable and satisfies 

We remark that the (in)equalities in Theorem 4.2 are indeed linear with respect to the design 
variables. The linearity with respect to the matrix variables W 7 , i?^, Fy^s, and Gy^s is obvious. 
The linearity with respect to v follows from ( 6 ). We also remark that the constraint (25) makes o a 
probability measure. 

Let us prove Theorem 4.2. 


( 22 ) 

(23) 

(24) 

(25) 


Proof of Theorem 4.2 

Assume that 7 > 0, Wx G Rx € M”^”, Fy^g G Gy^g G M"^”, and 0(^,6) satisfy 

(22)-(25), and let us define K by (19). In the same way as in the proof of Theorem 4.1, there 
exists an e > 0 such that Qx = VxiR) + el satisfies 



T{x)EK,xEK,y 

★ 


Ak.x^i.^ 
-iG. 




7,(5 


> 0 , 


VLv 


^K .x^ 1 
G^gQZ^G. 




7,5 


> 0 . 


(26) 

(27) 


Applying Schur complement to the matrix in the left hand side of (26), we obtain 
Rx — J1 {x)Ek.xGk,x — ■^K,xQx^li.x ^ ^PPlyiiig th® operator Vx to this inequality 
yields VxiAxQAj^ + fLEj^Ej.) < VxiR) < Qx- In addition, from (27) it follows that 
Wx> Gk.xQxGk^x- Hence, we have Exea: ^(‘^tf.xQxCA.x) < ^xea: ^(117) < 7 by (24). 
Therefore, by Proposition 3.4, Y^k is mean square stable and satisfies \\Ek\\1<1. □ 


Finally, the next theorem provides an LMl formulation to solve the iJoo control problem stated in 
Problem 2.11: 

Theorem 4.3 

Assume that 7 > 0, 77^ ^ K"""”, 26^ G M"^", Zx,x' G K"''”, Gy^g G M”""", and Fy^g G 
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= (a, -y, ( 5 ) and x' in satisfy the linear matrix inequalities 



-k 

k: 

k: 



0 

7 / 

Ea 

< 

1 

+ 

k 

k 

> 0 , 

(28) 

GctG-y^s H” § 

0 

0 

I 




^x,x' 

Hx 

* 1 > 0 , 



(29) 


for all X = (a, /3, 7 , 6) and x' in Define the feedback matrix K by (19). Then, the closed-loop 
system is mean square stable and satisfies ||Ei^||^ < 7 . 

Proof 

Assume that the inequalities (28) and (29) are satished by the matrices e e 

^x.x' e G-y.i G and G Dehne K by (19). Then, (28) implies 


-h 

k 

k 

k 

0 

7 / 

k 

k 

■^K,xG^,5 

Ek,x 


k 

GK,xG-i,5 

0 

0 

I 


By this inequality and (29), Proposition 3.5 immediately shows that Y^k is mean square stable and 
satisfies ||^ < 7 , as desired. □ 


5. NUMERICAL EXAMPLES 


The objective of this section is to illustrate Theorems 4.2 and 4.3 by numerical examples. We will 
also demonstrate how the periodicity of the feedback gain can be used to improve the performance 
of the closed-loop system Yx- 

Example 5.1 

In this example, we consider the Markov jump linear system studied in [12, Example 1]. The system 
has two modes and its parameters are given by 


- 0.7017 

-1.227 

0.3931 

-0.6368' 


■-0.0715 

-0.5420 

0.6716 

0.6250 ■ 

-0.4876 

-0.6699 

-1.7073 

-1.0026 

) ^2 — 

0.2792 

1.6342 

-0.5081 

-1.0473 

1.8625 

1.3409 

0.2279 

-0.1856 

1.3733 

0.8252 

0.8564 

1.5357 

. 1.1069 

0.3881 

0.6856 

-1.0540. 


. 0.1798 

0.2308 

0.2685 

0.4344 . 


Bi = Bo = 


h 

02x2 


, Cl =C2 = 


p = 


0.6942 

0.6942 


h 

02x4 

0.3058' 

0.3058 


, Di = D2 = 


04x2 

I 2 


, El = E 2 = I 4 , 


, ^JLr = [0.6942 0.3058] , 


where and Onxm denote the n x n identity matrix and the n x m zero matrix, respectively. 
Notice that the mode signal r is a sequence of independently and identically distributed random 
variables. 

We assume that the controller observes the mode through a Gilbert-Elliot channel, described in 
Example 2.2. Eor simplicity in our presentation, we let p = q. Notice that, whatever value p takes, the 
limiting distribution of s is the uniform distribution on the set { 1 , 2 }; in other words, the asymptotic 
frequency of the controller observing the mode signal r is 1 /2. In addition, the expected duration of 
the chain s staying at either Good or Bad state depends on p, and is equal to 1/p. We assume that 
the initial distribution ps of s is the uniform distribution. 

We can use Theorem 4.2 to design stabilizing feedback gains and the initial distribution v in order 
to achieve a small iT 2 norm of the closed-loop system Yk by solving the following optimization 
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Figure 2. The H 2 norm of the closed-loop system versus the expected duration 1 /p 


problem: 

minimize 7 , 

117 . -Rxi G^,5,!^(7,5) 

subject to (22)-(25). 

Figure 2 shows the H 2 norms of the optimized closed-loop systems. As expected, the larger the value 
of the period T, the smaller the attained H 2 norm. We can also see that the H 2 norm of the closed- 
loop system increases as the expected duration 1 jp increases, although the stationary distribution 
of s does not depend on p. We remark that this feature arising from the Markov property of s 
cannot be captured by the framework in [12], where mode observations at different time instants are 
assumed to be independent events with identical probabilities. 

Example 5.2 

Consider the Markov jump linear system S with the following parameters: 


^1 


■-0.6 

-0.4 


■-0.8 

0.4 


■-0.3 


■-0.2 

-0.6 

-0.4 

5 ^2 — 

0.8 

0.2 

; — 

-0.2 

, B 2 — 

-0.3 


Cl = [0.4 0.2] , C2 = [0.1 0.5] , Di = 0.1, D 2 = -0.3, 


0.1 0.9 
0.7 0.3 


El = 


-0.3 

-0.3 


E2 = 


- 0.2 

- 0.1 


P = 


From the standard theory of Markov jump linear systems [1], one can check that S is not mean 
square stable when u = 0. We use the observation process with independent and identically 
distributed failures (described in Example 2.3). In order to design stabilizing feedback gains 
achieving a small Hoc, norm of T,k, we solve the following optimization problem based on 
Theorem 4.3: 

minimize 7 , 

subject to (28) and (29). 

Figure 3 shows the iToo norms of the resulting closed-loop systems for various values of period T 
and failure probability p/. As we increase the period T, the Hao norm tends to decrease. However, 
notice that whenp/ is around 0.1, the El 00 norm attained by the controllers with T = 3 are better than 
those with T = 5. This phenomenon can happen because 5 is not an integer-multiple of 3. Remark 
that, for T = 6 instead of T = 5, such a phenomenon will not happen because the iToo performance 
obtained by the feedback gains = {Ar^^]}..yg[2],5G[3] with period T = 3 is attained by the 

feedback gains K^^'> with period T = 6 given as K^l = for every £=1,2,3. We 

also remark that, by the same reason, the performance of the resulting closed-loop system for any 
integer T is never worse than the performance for T = 1 (since 1 divides T). 
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Figure 3. The Hca norm of the closed-loop system versus the failure probability 


2 



-d-T = 1 
-o-T = 5 


/ ' 

1 


9 


0 ^ 

0 


\\ 




a 'v 



-a-D-- 




5 10 

k 


Figure 4. Sample averages of ||z(fc)||^ 


Finally, Figure 4 shows the sample averages of ||z(fc)|p of the closed-loop systems for T = 1 
and T = 5. For the computation of the sample averages, we fix xq = [1 2]^, ro = 1, sq = 1, and 

pj: = 0.5. The disturbance signal is chosen as w{k) = 2cos(fc/2). We generate 300 sample paths 
of r and s. Using the sample paths, we then generate 300 sample paths of z for T = 1 and T = 5, 
respectively. We can see that the closed-loop system with T = 5 attenuates the disturbance signal 
better than that with T =1. 


6 . CONCLUSION 

In this paper, we have studied state-feedback control of Markov jump linear systems with hidden- 
Markovian observations of the mode signals. This observation model generalizes various relevant 
cases previously studied in the literature on Markov jump linear systems, such as the cases with 
perfect information, no information and cluster observations of the mode signal. We have then 
developed an optimization framework, based on Linear Matrix Inequalities, to design feedback gains 
for stabilization, H 2 and Hao control problems. Finally, we have illustrated the effectiveness of this 
optimization framework with several numerical examples. 
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